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ABSTRACT 

The program review process in Kansas Regent's 
institutions was studied, based on Cuba's three policy types: 
policy-in-intention (statements about policy); 

policy* in- implementation (behaviors/activities found in the process 
of implementing policy); and policy-in-experience 
(constructions/perceptions of individuals as they experienced the 
policy enactment). The program review process of the University of 
Kansas School of Education was assessed and compared with processes 
at the Emporia State University and Kansas State University. Specific 
objectives were to: determine whether di icrepencies existed between 
policy-in-intention and policy-in-experience; identify factors in the 
implementation process that produced discrepancies; and recommend 
interventions that would reduce the dipcrepancy between the processes 
of intention, implementation, and experience. In addition to 
examining documents and records of the colleges, interviews were 
conducted with staff members. It was found that Cuba's policy types 
were helpful in viewing the program review process. Of: four policy 
intentions for conducting the program review, only one, strengthening 
the role of the Board of Regents, w£is not accomplished a.^ desired. 
Participants perceived the process differently depending on their 
role. Additional findings are covered. (SW) 
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Changes in Program Review Policy: A Case Study 



After several decades of growth and expansion, higher 
education is faced with static and declining budgets which 
appear to threaten academic quality and program 
flexibility. The declining budgets have raised several 
issues related to the role of higher education in society 
today. Included amonn these issues are the extent to which 
the public should support higher education, increased 
demands for accountability, the role of state and federal 
government in policy making for higher education, and the 
percentage of higher education funding which should be 
borne by the parents and students. Largely as a direct 
result of the accountability issue but related to issues of 
financial support for higher education, program review has 
come into vogue to assist colleges and universities in 
making budgetary and programmatic allocations. 

Concurrent with the increase of program review has been 
a flood of reports related to the quality of American 
education (Adler, 1982; Boyer, 1983; Feistritzer, 1983; 
Task Force on Federal Elementary and Secondary Education 
Policy, 1983; The National Commission on Excellence in 
Education, 1983; Goodlad, 1984; The National Institute of 
Education, 1984; The National Endowment for Humanities^ 
1984; Association of American Colleges, 1985; The National 
Commission for Excellence in Teacher Education, 1985). By 
far the most widely publicized of these reports has been A 
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Nation at Risk y prepared by the National Commission on 
Excellence in Education and published in April of 1983. The 
report is a study of the problems facing American education 
and it found that "the educational foundations of our 
society are presently being eroded by a rising tide of 
mediocrity that threatens our very future as a nation and a 
people" (p. 5). While this report and others have studied 
various aspects of the problems in the educational systems, 
virtually all the reports have indicated the need to 
improve the teacher education preparation provided by 
colleges and universities. 
Problem Statement 

Within this climate of increased accountability, 
declining resources, and national concern over the quality 
of the educational system, many states have initiated 
reviews to assist in determining priorities for academic 
programs. Evidence of this is seen in the involvement of 
state agencies in program reviews, in a 1981 study 
conducted for the National Center for Higher Education 
Management Systems (NCHEMS), Barak found thut fewer than 
five of the st?.ce coordinating/governing agencies had 
conducted reviews ten years ago. At the time of the study, 
twenty-eight state level agencies reported the authority to 
review existing programs and most were involved in some 
type of program approval. 
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The heightened awareness of boards and state agencies 
to the concept of program review can be attributed to 
several factors including enrollment imbalances caused by 
increased student interest in business and technology and 
limited growth or actual decreases in federal and state 
funds. Millard (1977) believed the growing interest in 
state level program review was related as much to 
legislative concern with minimum competency on the 
elementary/sc^condary level as to the enrollment shifts and 
retrenchment. 
Theoretical Framework 

The literature provided multiple and often conflicting 
definitions for policy. In fact, Klein and Murphy (1973) 
stated that the same word was often used to describe 
different concepts. They suggested that policy is better 
defined as a continuum moving from the very generalized 
statements about the purpose cf an organizat iOii to specific 
statements regarding rules in an organization. Cuba (1984) 
theorized that the word "policy" has multiple definitions 
and the type of definition applied in a situation 
"determines the kinds of policy questions that are asked, 
the kinds of policy-relevant data that are collected, the 
source of data that are tapped, the methodology that is 
used, and finally, the policy products that e.aerge" ( p. 
63). 
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According to Guba ( 1984), three policy types arc^ 
derived from eight basic definitions of policy found in the 
literature. It was these three policy 

types — policy-in-intention, policy-in-implementat ion , and 
policy-in-experience — that formed the theoretical base for 
*"he study. 

Policy-in-intenticn was defined by Guba (1984) as 
statements about policy. This type of policy is 
distinguished by assertion of intents or goals, standing 
goals, guides to discretionary actions and strategies to 
solve or ameliorate problems. Policy-in-implementation was 
characterized by behaviors or activities demonstrated in 
the process of implementing policy. Such behaviors or 
activities were defined by Guba (1984) as sanctioned 
behaviors, norms or conducts, and the cumulative output of 
the policy-making system. The third type of policy referred 
to by Guba was classified as the pol icy-in-exper ience and 
represented the constructions or perceptions of the 
individuals as they experienced the policy enactment. By 
examining the program review process from a variety of 
perspectives, the researcher sought to determine whether 
the types of policy as defined by Guba (1984) represented 
an appropriate theoretical framework in which to examine 
the program review process. 

Examining the dynamic and value-laden elements of 
processes such as program review is difficult because the 
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process is more akin to James Joyce's "stream of 
concciousnecs" than to a unitarily measurable and 
controllable action such as financial auais-s or 
distribution of head counts per major program. Policy 
studies have frequently been limited t-o examination of 
intentions or statements about policy and/or the results of 
such policies on various programs or institutions. The Guba 
(1984) framework provides for a method of examining policy 
in action and demonstrates the non-unitary, non-linear 
nature of policy. Furthermore, if found appropriate to 
program review processes, the Guba (1984) framework would 
provide an excellent construct for "stepping into a stream" 
of events and demonstrating that what is intended by 
policy-makers is often not what is implemented nor what is 
experienced by those participating in the policy processes. 

While there have been some studies related to the 
nature of program reviews (Melchiori, 1982), the use of 
reviews as retrenchment strategies (Mingle, 1981), and 
evaluation effectiveness of reviews (Baldridge, Kemerer and 
Green, 1982), little has been done to determine how the 
process of program review was actually applied in the field 
and how the process becomes modified and changed as a 
result of the social, economic, and political forces which 
impinged from the larger supra-^system. Therefore, this 
study focused on the differences in policy types (Guba, 
196-;) as they wef^ enacted through the program review 
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process conducted at the University of Kansas School of 
Education. The study then compared the results of the 
University of Kansas case with data collected at Emporia 
State University and Kansas State University to analyze the 
similarities and differences between policy enactments as 
defined by Cuba. 

Utilizing the case study approach, the researcher 
tested whether the policy-in-intent ion regarding program 
review differed from the policy-in-exper ience as perceived 
by the participants in the process and, where differences 
occurred, sought to identify the factors in the 
implementation process that produced such changes. 

The purpose of the study, therefore, was 1) to examine 
the program review process implemented in three 
institutions in a midwestern state to determine if 
discrepancies existed between policy-in-intent ion and 
policy-in-experience (Cuba, 1984); 2) to identify factors 
in the implementation process which produced discrepancies, 
where found; and 3) to recommend interventions in the 
process which would serve to reduce the rate of discrepancy 
between policy-in-intent ion, policy-in-implementat ion and 
policy-in-exper ience . 

Definitions and Limitations of the Study 

Many definitions of program review have appeared in 
the literature (examples of which are found in Folger, 
1977, and Arns and Poland, 1980). However^ the definition 
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which most appropriately represented the concept of program 
review examined in this study was that given by Seeley 
(1981) who defined program review as "a management and 
learning process of systematically identifying and 
collecting information about a set of related activities 
that have been developed to accomplish some end" (p, 45), 

Several classifications cf program review were also 
found in the literature. Harcleroad (1980) used the terms 
institutional and statewide while Barak (1985) identified a 
system dependent on the agency conducting the review, 
Howeverr for this study^ the classification internal and 
external mentioned by Baldridge, et al , r 1982 most 
appropriately described the type of review .studied. 

External reviews were those that were done voluntarily 
(usually to secure accreditations) or involuntarily (to 
satisfy legislative mandates or regents policy). These 
reviews have been characterized by standard data 
requirements r pre-established formats, and evaluations 
conducted by those outside of the university. Self-studies 
for regional or specialized aCcreditaLions and program 
reviews completed by governing bodies or state agencies 
were two examples of such studies. 

The subject of this study was an external review model 
formulated by the Kansas Board of Regents in December, 1982 
and applied simultaneously to the schools of education at 
three institutions in Kansas, The term external was applied 
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because the evaluation was conducted by an external body of 
consultants for the Board of Regents. The review included 
all levels of programs from the associate through the 
doctoral level and was designed to be cyclical so that 
every program is reviewed once every five years. For 
example, the first year of the review, four programs were 
scheduled. In fiscax year 1984, seven programs including 
education ware considered. The remaining programs are 
scheduled for fiscal years 1985-87. Each institution within 
the system with a program under leview is asked to prepare 
and submit clata regarding that program to the Board. The 
Board then uses those data to accomplish the following 
Objectives : 

(1) Further strengthen the role of the Board of 
Regents in governing the institutions under its 
jurisdiction; 

(2) Increase the Board's knowledge and understanding 
of the programs conducted at its institutions in 
order to make informed decisions In an ever- 
changing environment; 

(3) Provide for the periodic and systemwide review of 
all programs at each Regents insitution; 

(4) Provide for additional self-study of all programs 
in order co improve management at the 
institutional level (Minutes, Kansas Board of 
Regents, December 17, 1982). 
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Research Methods 

The research employed the case study method, incluf^ing 
non-participant observation, interviews and document review 
in gathering the data, in oidcr to increase the likelihood 
that the widest range and scope of data would be uncovered, 
purposive or theoretical sampling was used. The three 
institutions chosen to be sampled in the study were 
selected because they represented critical cases. According 
to Pattnn (1980), using the sampling strategy ot critical 
cases allows "logical generalization ^nd maximum 
application of information to other cases because if it's 
true of this one case, it's likely to be true of all other 
cases" (p. 105). 

Both the University of Kansas and Kansas State 
University were chosen because they are comprehensive 
research institutions with highly visible and politically 
sensitive schools of education. Kansas State University has 
as its primary mission extension and service to the state 
while the University of Kansas was noted for its research 
and the five-year extended teacher education program. 
Emporia state University was chosen because it had a large 
teacher education program and has gained recognition for 
strong programs in the areas of applied research to 
education and in-service programs for the public school 
sector . 
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The documents and records that were evaluated in this 
stud/ v^ere examined and collected over an eleven-month 
period while the researcher was on-sit'i at one of the 
locations. Content analysis was employri as the method for 
the analysis of documents and records. "Content analysis is 
a research technique for the objective, systematic, and 
quantitative descr ipti^^*^ of the manifest content of 
communication" (Cuba and Lincoln, 1981, p. 240). 

Interviews were conducted with forty-eight individuals 
including faculty, staff, anJ adrf^inistrators ac the three 
institutions, selected members an^i former members of the 
Kansas Board of Regents, Board of Regents st,aff, and 
legislators who had participated in the program review. 
Sin^e tie basis of the inquiry was to search for the 
multiple perceptions regarding the process of pL'ogram 
review, non-standardized, open-ended interviews were used. 

The coding procedure for t\\is study was initially 
determined by the research questions. These categories 
included types of policy statements, political, social or 
-economic issues, process statements, centrality, quality 
and socitital demand issues, and characteristics of internal 
and external models of review. As the study progressed, new 
categories were added and additional sorts were made. At 
the conclusion of the study, sixteen categories were used 
for the analysis of the data. Data analysis was performed 
simultaneously with data collection by adapting a commonly 
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used computer program Multiplan to the task of corting and 
retrieving the data. The researcher found that by using the 
computer as a tool to manage, retrieve and organize the 
data into a variety of patterns, the research was greatly 
facilitated, Conrad and Reinharz (1984) recognized this 
advantage and maintained that "by taking over mc ny of the 
mechanical aspects of qualitative analysis, the computer 
allows the researcher to devote more of his or her energy 
to the interpretative or analytic work, which is more 
significant and rewarding" (1984, p, 9), 
Results and T)iscussion 

The Kansas Board of Regents had four goals for 
conducting the program review which formed the 
policy-of-intent. Tne goals, which we^e detailed earlier, 
c .sisted of 1) strengthening the role of the Board of 
Regents in governing the institutions; 2) increasing the 
Hoard's knowledge and understanding of the programs? 3) 
providing for a systemwide review of all programs; and 4) 
providing for additional self-studies to improve 
institutional management. 

ThG research indicated that goals two, three and four 
were readily fulfilled through the review process, in fact, 
the goal of Board education was cited by many participants 
as being one of the most import--- ^spects of the review 
process. As one Board staff member said, "Just being on 
campus and experiencing the academic lifa is a positive 

14 
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aspect of the review for the Regents. There was a certain 
value in talking to the- facultyr understanding that they 
are real people and not so bad after all. It helps take 
away the myth" (Aprils 1985). 

Certainly the Board succeeded in fulfilling goal three 
of providing a systemwide process for reviewing programs. 
Furthermore^ the fourth goal of improving institutional 
management was realized oy the increased institutional 
awareness for the need for accountability and credibility. 
Tiie revxew process itself sensitized the campuses to the 
political side of academic life and i5ome of the 
participants felt that this resulted in better management 
of the institution. "I believe the institutions* 
administrations are learning more about the programs than 
they ever have. They have to do this because they have to 
come beCore the Board and defend their procjtams" 
.Administrator r April, 1985). 

The study found, however, that the first goal of 
strengthening the role of the Board of Regents in governing 
the institutions was not accomplished to the degree 
intended. An analysis of the data revealed three factors 
which resulted in discrepancies and differences between 
what was intended initially and what the participants 
experienced. 

First, several members of the Board commented that 
they failed initially ho appreciate the complexities of the 
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academic programs, especially in regard to the education 
programs. One Board member acknowledged that the "review 
process pointed out how big and how complex the 
institutions were" (April, 1985). A former Board member was 
more direct when stating their difficulties: "It was very 
confusing for the Board members try to sort out the 
complexities of the various programs and we were confused 
about the nature of the programs. ... if they (Board 
members) couldn't know something entirely or 100%, i was 
not sure whether they should be making t^ose types of 
decisions" (May, 1985). 

These comments reflect the business orientation of the 
Board but do not reflect the realities of higher education. 
Higher education is an enterprise which deals with people 
and with bodies of knowledge wMch cannot be measured by 
profit-loss statements nor can be ever known entirely or 
100 per cent. 

The education reviews were the first time the Regents 
were asked to look at programs which involved all of their 
institutions and to review so many different program areas. 
It was partially in recognition of this complexity that the 
Board retained consultants to conduct the reviews, since 
consultants are frequently used in program review 
processes, it is a note of some importance that their 
effect on the policy-in-implementation be considered. The 
Board had hoped that the consulcar.Ls would provide 
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recommendations for makinj some changes in the 
configurations of the education programs. They were, 
however , disappointed, and the consultants report focused 
almost entirely on graduate programs and failed to make any 
recommendations regarding teacher education programs, the 
original intent of the review. In the words of one Board 
member, "I had really hoped that the consultc^nts would be 
able to come and make some substantial, significant, or 
radical recommendations regarding the configuration of the 
schools of education. But the report lacked significance 
and they really didn't say anything different than what the 
Regents would have found had they conducted the review 
themselves" (April, 1985). 

Finally, the Board found that when it discussed making 
substantial changes in the program offerings, there were 
always tnose who opposed such changes and who would create 
political pressure on the Board to prevent action. It was 
the political action more than anything else which 
prevented the Board from fulfilling its intended policy 
goal. Participants in the process ruefully described the 
effects of the political action. One faculty member noted, 

. . that once politics enter into the decision-making 
process you have to ignore the data. That you have the 
information about which programs were good and bad, whi'-:h 
were strong or weak, but politically they Cthe Board] could 
not act on it* (May, 1985). This opinion was reinforced by 
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a Board of Regents staff member: " if I could^ T would try 
to totally do away with any political considerations ^ 
political being in the large senee of the legislature and 
the governor^ because the political processes caused 
knee-jerk type reactions and it is almost impossible to be 
objective about things when they are in the political 
environment" (May^ 1985) . 
ftssumptions and Interpretations: 

The process of program review was viewed by board 
members^ staff and administrators as a means of limiting 
t .3 political effects of decision-making on those in 
positions of power and authority. Many believed that the 
review process would gather and make available so much 
information that a consensus regarding future actions would 
be generated and^ therefore^ reduce or eliminate the 
political nature of program decisions. In actuality^ the 
review process did increase the flow of information 
regarding these decisions^ but it also heightened the 
attention on decisions^ and^ in party acted to increase the 
politicization of the process. Consensus was not 
forthcoming but rather turf battles erupted which further 
increased the politics of the process. The program review 
process did not remove the politics from the decision 
making arena. In fact^ the political environment is not 
only inescapable^ it is so pervasive t-hat one ought to 
expect its influence to be felt every step of the way in 



ERLC 



18 



Changes in Program Review 16 
such a process. The Board only happens to operate closer to 
the seat of the largest political influence — the 
legislature and the novernor's office; but the institutions 
themselves are also a part of the system of systematic 
influences which "shape" the system A, in turn, are 
shaped by it. 

This is why, in this instance, and in many other 
instances, policy-in-intent becomes a new creature — 
policy-in-implementation — whi'^h then metamorphoses into an 
entirely new creature — policy-in-experience , One cannot 
escape the influence of politics, because politics is an 
expression of the one or several value systems which 
operate in any given environment. 
Conclusions and Interventions 

By studying the program review process in the Kansas 
Regent's institutions, the researcher was able to observe 
changes in the process over time. The following paragraphs 
review these observations in order to determine if the Cuba 
policy types are useful as a theoretical framework for 
understanding program review. In addition, some conclusions 
are drawn about how the process was experienced by the 
different participants. Finally, the paper concludes with a 
series of interventions or recommendations which may be 
useful in affecting changes in the process in the future. 
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Three Policy Types 

Cuba's three policy types (1984) reflected an accurate 
representation of the manner in which the policy of program 
review was viewed by the participants. In the beginning the 
policy-in-intent statements^ these statements represented 
by the official policy statements approved by the Board in 
December^ 1982^ provided the basis on which the process was 
structured. In general^ all participants accepted those 
goals and objectives^ and data were collected to support 
such goals. For example^ in the early stages of the review 
process^ consideration was given to having all the 
institutions use the ETS surveys employed by the University 
of Kansas graduate school. However^ during the 
implementation stage of the process^ various participants 
began to make changes in the process^ usually to achieve 
some other purpose than the ones stated in the original 
policy statements. 

Nevertheless y it was only when the researcher 
interviewed the various participants in the process that 
the differences in policy-in-experience were noted. It was 
found that each individual^ based on his or her own 
experience^ background and organizational perspective^ had 
assigned different values and meanings to the various 
activities in the process. Therefore^ what the Board had 
intended as the goals of the process ware altered by both 
the intervening policies of the implementation stage and 
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the individual attitudes of the participants as they 
experienced the policy actions. Th^ process was perceived 
differently institutionally a;id individually. The 
recognition and awareness that policy does change over time 
is an important factor to be considered by both governing 
boards and administrations. The Cuba description of policy 
types would suggest that policy will change from intent to 
implementation to experience and that such changes should 
be expected by those involved in the program review 
process. 

Interventions 

Based on the data and the conclusions drawi*, the study 
indicated that there were some strategies or steps that 
could be taken to gain maximum benefit from the review 
process and to shape changes in the process as it is 
implemented. 

In the previous sectionr it was noted that 
participants perceived the process differently depending on 
their organizational role. This evidence supports the need 
for better coiamunication among participants and 
organizational unitSr a need mentioned by many of uhe 
participants. Thereforer the following interventions aie 
provided to improve communication links. 

Pirstr the process should allow for sufficient time 
for proper implementationr evaluation and review. When the 
program review process was hurriedr as in the 'eacher 
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education review^ it caused breakdowns in the process which 
led to frustration and misunderstandings. Possible methods 
suggested for providing more time included reviewing fewer 
programs each year. For example, the Board could review 
four per six month period co allow more in-depth review. 
Another suggestion was to devise a method for Ldentifying 
those programs who were in stress and then to review only 
those programs extensively. The time required by the Board 
for conducting the reviews has become quite burdensome, and 
this factor in itself caused concern. If service on the 
board requires a significant time co.wmitment, such as 
seventy-five days which one board member indicated had been 
spent during one year of the program reviews, eventually 
only the wealthy will be able to serve. There is a 
disturbing philosophy inherent in that kind of service. 

The participants in the process also reed more 
structured ways of sharing information and viewpoints. By 
recognizing the c»ffect of one's role on process 
perceptions, steps can be taken to inform the various 
participants of other's viewpoints so they may be aware of 
the larger context of the review process. Such steps would 
serve to improve communication between the institution and 
the Board. Institutional representatives could be included 
in the designing of the process, and board members and 
board staff could be invited to address faculty and student 
groups. One institution in the study had recently begun 
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such an effort through its faculty senate. The senate 
president also saw that all faculty members received copies 
of the Board's minutes. Another suggestion offered by many 
was to develop a comparable data base. This would not only 
improve communication between institutions and board but 
between institutions as well. 

Finally, the Board should minimize the perceived 
threat to institutional autonomy where possible. Those 
charged with resigning and implementing the program review 
process, both at the state and institutional level, should 
take steps to protect the academic and administrative 
perogatives of faculty and administration, suoh steps could 
include providing funding for the self-studies so that the 
institutions feel that the Board and institutions are 
partners in the process rather than adversaries. Secondly, 
consultants should be selected more for their familiarity 
with regional trends and state needs than for cheir 
national prominence. Ineffective consultants harm the 
credibility of the process for everyone involved. Finally, 
some consideration should be given for putting the review 
back in the hands of the institutions. By doing this, the 
institutions would tend to be more critical of themselves, 
and the Board could monitor the reviews for quality and 
integrity. 

In summary, the study found the policy types as 
defined by Guba (1984) to be an appropriate theoretical 
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framework for /iewing the program review process. The Board 
had four policy intentions for conducting the program 
review. With regard to the goals of increasing Board 
knowledge and understanding, providing a systemwide review, 
and providing a self-study to improve management, the Board 
achieved its goals. However, in strengthening their role in 
governing the institutions, the Board was unable to make 
substantial changes in any of the education programs. The 
Board's failure to achieve its intention was due to three 
factors—the complexities of the education programs under 
review, the failure of the consultants to provide credible 
recommendations for action and the political environment in 
which the review operated. The study found that the role of 
the participants in the organization affected the 
participants experience in the process. In addition, in 
spite of efforts by the Board to recognize institutional 
autonomy, the process was seen as a threat to traditional 
academic and administrative perogatives. Interventions were 
suggested to improve the process for future use. 

Further research should be conducted to determine if 
organizational role is a predictor of attitude regarding 
the program review process and to what extent policies 
change during the implementation and experience stages, 
such research would bs valuable in determining how policies 
are altered in the implementation process, and what, if 
any, changes in policy enactments are needed. 
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